Developing an explainable deep learning boundary correction method by incorporating cascaded x-Dim models to improve segmentation defects in liver CT images.

Deep learning methods achieved remarkable results in medical image analysis tasks but it has not yet been widely used by medical professionals. One of the main reasons for this restricted usage is the uncertainty of the reasons that influence the decision of the model. Explainable AI methods have been developed to improve the transparency, interpretability, and explainability of the black-box AI methods. The result of an explainable segmentation method will be more trusted by experts. In this study, we designed an explainable deep correction method by incorporating cascaded 1D and 2D models to refine the output of other models and provide reliable yet accurate results. We implemented a 2-step loop with a 1D local boundary validation model in the first step, and a 2D image patch segmentation model in the second step, to refine incorrect segmented regions slice-by-slice. The proposed method improved the result of the CNN segmentation models and achieved state-of-the-art results on 3D liver segmentation with the average dice coefficient of 98.27 on the Sliver07 dataset.

Reducing reconstruction error of classified textural patches by integration of random forests and coupled dictionary nonlinear regressors: with applications to super-resolution of abdominal CT images.

PURPOSE: Random forests and dictionary-based statistical regressions have common characteristics, including non-linear mapping and supervised learning. To reduce the reconstruction error of high-resolution images, we integrate random forests and coupled dictionary learning. METHODS: Textural differences of image blocks are considered by the classification of patches using an Auto-Encoder network. The proposed algorithm partitions an input LR image by 5 × 5 blocks and classifies training patches into six categories. A single random forest regressor is then trained corresponding to each class. The output of an RF is considered as an initial estimate of the HR slice. If a slice's representation is sparse in the Discrete Cosine Transform domain, the initial reconstructed image is further improved by a coupled dictionary. RESULTS: In this study, we applied our method to abdominal CT scans and compared them to conventional and recent researches. We achieved an average improvement of 0.06 (2.37) using the SSIM (PSNR) index compared to the random forest + dictionary learning method. CONCLUSION: The low standard deviation of the results reveals the stability of the proposed method as well. The proposed algorithm depicts the effectiveness of classifying image patches and individual treatment of each class.

Incorporating prior shape knowledge via data-driven loss model to improve 3D liver segmentation in deep CNNs.

PURPOSE: Convolutional neural networks (CNNs) have obtained enormous success in liver segmentation. However, there are several challenges, including low-contrast images, and large variations in the shape, and appearance of the liver. Incorporating prior knowledge in deep CNN models improves their performance and generalization. METHODS: A convolutional denoising auto-encoder is utilized to learn global information about 3D liver shapes in a low-dimensional latent space. Then, the deep data-driven knowledge is used to define a loss function and combine it with the Dice loss in the main segmentation model. The resultant hybrid model would be forced to learn the global shape information as prior knowledge, while it tries to produce accurate results and increase the Dice score. RESULTS: The proposed training strategy improved the performance of the 3D U-Net model and reached the Dice score of 97.62% on the Sliver07-I liver dataset, which is competitive to the state-of-the-art automatic segmentation methods. The proposed algorithm enhanced the generalization and robustness of the hybrid model and outperformed the 3D U-Net model in the prediction of unseen images. CONCLUSIONS: The results indicate that the incorporation of prior shape knowledge enhances liver segmentation tasks in deep CNN models. The proposed method improves the generalization and robustness of the hybrid model due to the abstract features provided by the data-driven loss model.

VesselNet: A deep convolutional neural network with multi pathways for robust hepatic vessel segmentation.

Extraction or segmentation of organ vessels is an important task for surgical planning and computer-aided diagnoses. This is a challenging task due to the extremely small size of the vessel structure, low SNR, and varying contrast in medical image data. We propose an automatic and robust vessel segmentation approach that uses a multi-pathways deep learning network. The proposed method trains a deep network for binary classification based on extracted training patches on three planes (sagittal, coronal, and transverse planes) centered on the focused voxels. Thus, it is expected to provide a more reliable recognition performance by exploring the 3D structure. Furthermore, due to the large variety of medical data device values, we transform a raw medical image into a probability map as input to the network. Then, we extract vessels based on the proposed network, which is robust and sufficiently general to handle images with different contrast obtained by various imaging systems. The proposed deep network provides a vessel probability map for voxels in the target medical data, which is used in a post-process to generate the final segmentation result. To validate the effectiveness and efficiency of the proposed method, we conducted experiments with 20 data (public datasets) with different contrast levels and different device value ranges. The results demonstrate impressive performance in comparison with the state-of-the-art methods. We propose the first 3D liver vessel segmentation network using deep learning. Using a multi-pathways network, segmentation results can be improved, and the probability map as input is robust against intensity changes in clinical data.

Balancing the data term of graph-cuts algorithm to improve segmentation of hepatic vascular structures.

PURPOSE: The accurate delineation of hepatic vessels is important to diagnosis and treatment planning. To improve the segmentation of these vessels and extract small structures, we adaptively calculate the data term in conventional graph-cuts algorithm. METHOD: To assign higher costs to the data term in small vessel regions, we estimate the statistical parameters of the vessel adaptively. After preprocessing an input CT image, we model the liver and its vessels by two Gaussian distributions. The Maximum Intensity Projection (MIP) of the image is employed in the Expectation-Maximization algorithm to estimate the parameters of the model. These parameters are used together with a medial-axes enhancement algorithm to find the axes of the vessels. The skeleton of these vessels is considered to be the image voxels that are most similar to the hepatic vascular structures. To calculate the cost function of the graph-cuts algorithm, those axes that are nearby are employed to estimate the vessel parameters. The conventional minimum-cut/maximum-flow energy minimization framework finds the global minimum of the cost function and labels vessel voxels. RESULT: We evaluated our method using synthetic data and clinical images. We compared our algorithm with state-of-the-art vessel segmentation methods. The mean Dice measure of our results was 95.51% (0.9% lower than the first rank method). Quantitatively, our method segmented small hepatic vessels that were not extracted by traditional techniques including conventional graph-cuts. CONCLUSION: The proposed method improved the segmentation of small vessels in the presence of noise.

A generalized active shape model for segmentation of liver in low-contrast CT volumes.

PURPOSE: To improve segmentation of normal/abnormal livers in contrast-enhanced/non-contrast CT image using the Active Shape Model (ASM) algorithm; we introduce a generalized profile model. We also intend to accurately detect boundary of liver where it touches nearby organs with similar intensities. METHOD: Initial boundary of a liver in a CT slice is found using an intensity-based technique and it is then represented by a set of points. The profile of a boundary point is represented by a generalized edge model and the parameters of the model are obtained using a non-linear fitting scheme. The estimated parameters are used to classify boundary points into genuine and dubious groups. The genuine points are located as the true border of the liver and the locations of dubious points are refined using smoothed spline interpolation of genuine landmarks. Finally, the liver shape is kept inside the "Allowable Shape Domain" using a Statistical Shape Model. RESULT: We applied the proposed method on four sets of CT volumes including low/high-contrast, normal/abnormal and public datasets. We also compared the proposed algorithm to conventional and state-of-the-art liver segmentation methods. We obtained competitive segmentation accuracy with respect to recent researches including enhanced versions of the ASM. Concerning conventional Active Shape and Active Contour models, the proposed method improved Dice measure by at least 0.05 and 0.08 respectively. Regarding the MICCAI dataset, we promoted our score from 68.5 to 72.1. CONCLUSION: The proposed method alleviates segmentation problems of conventional ASM including inaccurate point correspondences, generalization ability of the model and sensitivity to initialization. The proposed method is also robust against leakage to nearby organs with similar intensities.

Improved segmentation of low-contrast lesions using sigmoid edge model.

PURPOSE: The intensity profile of an image in the vicinity of a tissue's boundary is modeled by a step/ramp function. However, this assumption does not hold in cases of low-contrast images, heterogeneous tissue textures, and where partial volume effect exists. We propose a hybrid algorithm for segmentation of CT/MR tumors in low-contrast, noisy images having heterogeneous/homogeneous or hyper-/hypo-intense abnormalities. We also model a smoothed noisy intensity profile by a sigmoid function and employ it to find the true location of boundary more accurately. METHODS: A novel combination of the SVM, watershed, and scattered data approximation algorithms is employed to initially segment a tumor. Small and large abnormalities are treated distinctly. Next, the proposed sigmoid edge model is fitted to the normal profile of the border. The estimated parameters of the model are then utilized to find true boundary of a tissue. RESULTS: We extensively evaluated our method using synthetic images (contaminated with varying levels of noise) and clinical CT/MR data. Clinical images included 57 CT/MR volumes consisting of small/large tumors, very low-/high-contrast images, liver/brain tumors, and hyper-/hypo-intense abnormalities. We achieved a Dice measure of [Formula: see text] and average symmetric surface distance of [Formula: see text] mm. Regarding IBSR dataset, we fulfilled Jaccard index of [Formula: see text]. The average run-time of our code was [Formula: see text] s. CONCLUSION: Individual treatment of small and large tumors and boundary correction using the proposed sigmoid edge model can be used to develop a robust tumor segmentation algorithm which deals with any types of tumors.

Segmentation of liver and spleen based on computational anatomy models.

Accurate segmentation of abdominal organs is a key step in developing a computer-aided diagnosis (CAD) system. Probabilistic atlas based on human anatomical structure, used as a priori information in a Bayes framework, has been widely used for organ segmentation. How to register the probabilistic atlas to the patient volume is the main challenge. Additionally, there is the disadvantage that the conventional probabilistic atlas may cause a bias toward the specific patient study because of the single reference. Taking these into consideration, a template matching framework based on an iterative probabilistic atlas for liver and spleen segmentation is presented in this paper. First, a bounding box based on human anatomical localization, which refers to the statistical geometric location of the organ, is detected for the candidate organ. Then, the probabilistic atlas is used as a template to find the organ in this bounding box by using template matching technology. We applied our method to 60 datasets including normal and pathological cases. For the liver, the Dice/Tanimoto volume overlaps were 0.930/0.870, the root-mean-squared error (RMSE) was 2.906mm. For the spleen, quantification led to 0.922 Dice/0.857 Tanimoto overlaps, 1.992mm RMSE. The algorithm is robust in segmenting normal and abnormal spleens and livers, such as the presence of tumors and large morphological changes. Comparing our method with conventional and recently developed atlas-based methods, our results show an improvement in the segmentation accuracy for multi-organs (p<0.00001).

Multimodality liver registration of Open-MR and CT scans.

PURPOSE: Multimodality registration of liver CT and MRI scans is challenging due to large initial misalignment, non-uniform MR signal intensity in the liver parenchyma, incomplete liver shapes in Open-MR scans and non-rigid deformations of the organ. An automated method was developed to register liver CT and open-MRI scans. METHODS: A hybrid registration algorithm was developed which incorporates both rigid and non-rigid methods. First, large misalignment of input CT and Open-MR images was compensated by intensity-based registration. Maximum intensity projections (MIPs) of CT and MR data were registered in 2D, and the corresponding rigid transform parameters were used to align 3D images in axial, coronal and sagittal planes. Use of MIP projections compensates for intensity inhomogeneities inherent in the Open-MR data. A bounding box of MIP images defines an ROI which removes outliers and copes with incomplete MR data. Next, principal components analysis (PCA) was used to align MR and CT data datasets. The corresponding translation and rotation parameters were then used to increase the global registration accuracy. A modified TPS-RPM point-based non-rigid algorithm was used to accommodate local liver deformations. Surface points on the liver and branching points of the portal veins were input as landmarks to TPS-RPM method. Incorporating vascular branching points improves registration since tumors are usually found near vessels, so greater weight was given to branching points compared with surface points. RESULTS: The automated registration algorithm was compared with both rigid and non-rigid methods. Quantitative evaluation was performed using modified Hausdorff distance and overlap measure. The mean modified Hausdorff distances of liver and tumor were decreased from 23.53 and 40.03 mm to 9.38 and 8.88 mm, respectively. The mean overlap measures of liver and tumor were increased from 39 and 0 % to 78 and 27 %, respectively. Statistical analysis of the outcomes resulted in a p value less than 5 %. CONCLUSION: MIP-PCA-based rigid multimodality CT-MRI registration of liver scans compensates for large misalignment of input images even when the data are incomplete. A modified TPS-RPM algorithm, in which vascular points are emphasized over surface points, successfully handled local deformations.

Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: the VESSEL12 study.

The VESSEL12 (VESsel SEgmentation in the Lung) challenge objectively compares the performance of different algorithms to identify vessels in thoracic computed tomography (CT) scans. Vessel segmentation is fundamental in computer aided processing of data generated by 3D imaging modalities. As manual vessel segmentation is prohibitively time consuming, any real world application requires some form of automation. Several approaches exist for automated vessel segmentation, but judging their relative merits is difficult due to a lack of standardized evaluation. We present an annotated reference dataset containing 20 CT scans and propose nine categories to perform a comprehensive evaluation of vessel segmentation algorithms from both academia and industry. Twenty algorithms participated in the VESSEL12 challenge, held at International Symposium on Biomedical Imaging (ISBI) 2012. All results have been published at the VESSEL12 website http://vessel12.grand-challenge.org. The challenge remains ongoing and open to new participants. Our three contributions are: (1) an annotated reference dataset available online for evaluation of new algorithms; (2) a quantitative scoring system for objective comparison of algorithms; and (3) performance analysis of the strengths and weaknesses of the various vessel segmentation methods in the presence of various lung diseases.